Applying Language Model into IR Task

The language modeling approach to information retrieval is a new framework that has been proposed and developed within the past five years. In NTCIR4, we focus our experiments on evaluating the effectiveness of the LM based IR method. In C-C run, we observe that the average precision of two-stage smoothing language modeling IR increase about 10.09% compared with VSM method when “DESC” field is used as query while decrease nearly 7.81% when query is “TITLE” field. This proved that the two-stage language modeling IR method could increase the performance in longer query effectively. In E-C run, the average precision of two-stage smoothing LM IR decreases about 38.09% compared with VSM method when “DESC” field is used as query while decreases nearly 32.60% when query is “TITLE” field. We think the two-stage smoothing LM IR method is more sensitive to the noise words introduced by wrongly translated query than SVM method.